We claim: 

I . A video detection apparatus adapted to detect and follow movement of a 
predefined object of interest within the video data of the output of a video camera 
encoded in frame-wise data, each frame occurring at sequential time intervals, and 
including the segmentation of at least one portion of the frame-wise data, an object 
segment, as potentially including the preselected object of interest, and including 
vertical edge pixels and horizontal edge pixels detected to be present within the 
object segment, comprising: 

an edge symmetry detection unit comprising: 

a vertical symmetry detector adapted to compute the vertical symmetry of at 
least one of the set of vertical edge pixels and the set of horizontal edge 
pixels in each column about a horizontal axis passing through the object 
segment, and to produce a vertical symmetry value for columns within the 
object segment; 

a horizontal symmetry detector adapted to compute the horizontal 
symmetry of at least one of the set of vertical edge pixels and set of 
horizontal edge pixels in each row about a vertical axis passing through the 
object segment, and to produce a horizontal symmetry value for rows 
within the object segment; and, 

an edge symmetry feature set computation apparatus adapted to compute an 
edge symmetry feature set for the object segment based upon the vertical 
symmetry values and the horizontal symmetry values; and, 
a background modeling unit comprising: 

a background modeling apparatus adapted to isolate from a current frame at 
least one segment of the current frame, each of the at least one segments 
containing a possible object of interest within the current frame, on a frame- 
wise basis, and to provide as an output each of the at least one segments; 
an object of interest classifying apparatus adapted to determine whether or 
not any segment of the at least one segments output from the background 
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modeling apparatus includes a object of interest and to characterize any 
such segment as an object segment, utilizing at least in part the edge 
symmetry feature set; 

an object segment tracking apparatus adapted to track the location within 
the current frame of any object segment, and to determine a projected 
location of the object segment in a subsequent frame, and to provide the 
background modeling apparatus with the projected location in the 
subsequent frame of the object segment; and 

wherein the background modeling apparatus is adapted to provide as its 
output the projected location of the object segment in the subsequent frame 
in lieu of a segment from the subsequent frame determined by the 
background modeling apparatus to contain a possible object of interest or in 
the event that the background modeling apparatus is unable in the 
subsequent frame to locate any segment containing a possible object of 
interest. 

2. The apparatus of claim 1 , further comprising: 

the edge symmetry unit further comprising: 

a vertical symmetry detector adapted to compute the vertical symmetry of 
the set of vertical edge pixels and the set of horizontal edge pixels in each 
column about a horizontal axis passing through the object segment, and to 
produce a vertical symmetry value for columns within the object segment; 
a horizontal symmetry detector adapted to compute the horizontal 
symmetry of set of vertical edge pixels and set of horizontal edge pixels in 
each row about a vertical axis passing through the object segment, and to 
produce a horizontal symmetry value for rows within the object segment; 
a feature set computation apparatus adapted to compute a feature set for the 
object segment based upon the vertical symmetry values and the horizontal 
symmetry values. 
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3. The apparatus of claim 2, further comprising: 

the object segment tracking apparatus further comprising: 

a matching apparatus adapted to match the location of an object segment in 

the current frame to one of a plurality of projected locations of the object 

segment in the current frame, which projections are based upon the location 

of the respective object segment in at least one prior frame; and, 

a track provider adapted to receive and store the location of an object 

segment in the current frame and over a plurality of prior frames and 

adapted to provide the projections of the location of the object segment in a 

subsequent frame based upon the locations of the object segments in the 

current frame and the plurality of prior frames. 

4. The apparatus of claim 3, further comprising: 

the vertical symmetry detector is adapted to compute the vertical symmetry 
value of at least one of the set of horizontal edges or the set of vertical edges in a 
given column / as the ratio of the number of horizontal edge pixels or vertical edge 
pixels, respectively, in each column / on opposing sides of the horizontal axis, 
independent of which side has more horizontal edge pixels or vertical edge pixels, 
respectively, expressed as a number between 0, if there are no horizontal edge 
pixels or no vertical edge pixels, respectively, on either side of the horizontal axis 
and 1 .0 if there are, respectively, an equal number of horizontal edge pixels or 
vertical edge pixels, respectively on both sides of the horizontal axis; 

the horizontal symmetry detector is adapted to compute the horizontal 
symmetry value of at least one of the set of horizontal edges or the set of vertical 
edges in a given row j as the ratio of the number of horizontal edge pixels or 
vertical edge pixels, respectively, in each row j on opposing sides of the vertical 
axis, independent of which side has more horizontal edge pixels or vertical edge 
pixels, respectively, expressed as a number between 0, if there are no horizontal 
edge pixels or no vertical edge pixels, respectively, on either side of the horizontal 
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axis and 1 .0 if there are, respectively, an equal number of horizontal edge pixels or 
vertical edge pixels, respectively on both sides of the vertical axis. 

5. The apparatus of claim 4, further comprising: 

the background modeling apparatus is adapted to provide as its output the 
projected location of the object segment in the subsequent frame in order to 
override any conflict in the labeling of pixels in the subsequent frame as part of a 
segment containing a possible object of interest. 

6. The apparatus of claim 5, further comprising: 

the feature set computation apparatus further comprising: 

a horizontal fuzzy set computation apparatus adapted to compute a fuzzy 

set value for each of a plurality of horizontal fuzzy sets each having a 

selected weighting function applicable to a selected group of columns 

within the object segment, with the columns grouped according to position 

along the horizontal axis in the object segment; and 

a vertical fuzzy set computation apparatus adapted to compute a fuzzy set 

value for each of a plurality of vertical fuzzy sets each having a selected 

weighting function applicable to a selected group of rows within the object 

segment, with the rows grouped according to position along the vertical 

axis in the object segment. 

7. The apparatus of claim 6, further comprising: 

the background modeling apparatus is adapted to isolate from a current 
frame the at least one segment of the current frame utilizing the video object plane 
extracted from the current frame. 

8. The apparatus of claim 7, further comprising: 

the horizontal fuzzy sets further comprising: 

a left extreme fuzzy set having a weighting function that is a constant value 
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for a first selected number of columns from the left border of the object 
segment toward the vertical axis of the object segment and decreases to 
zero over a second selected number of columns toward the vertical axis if 
the object segment; 

5 a right extreme fuzzy set having a weighting function that is a constant 

value for a third selected number of columns from the right border of the 
object segment toward the vertical axis of the object segment and decreases 
to zero over a fourth selected number of columns toward the vertical axis of 
the object segment; and 
10 at least one horizontal internal fuzzy set having a weighting function that 

increases from zero to a peak value and decreases to zero as the columns 
contained in each such internal fuzzy set go from left to right across the 
object image; and, 

the vertical fuzzy sets further comprising: 

15 a bottom extreme fuzzy set having a weighting function that is a constant 

value for a fifth selected number of rows from the bottom border of the 
object segment toward the horizontal axis of the object segment and 
decreases to zero over a sixth selected number of rows toward the 
horizontal axis if the object segment; 

2 0 a top extreme fuzzy set having a weighting function that is a constant value 

for a seventh selected number of columns from the top border of the object 
segment toward the horizontal axis of the object segment and decreases to 
zero over an eighth selected number of rows toward the horizontal axis of 
the object segment; and, 

2 5 at least one vertical internal fuzzy set having a weighting function that 

increases from zero to a peak value and decreases to zero as the rows 
contained in each such vertical internal fuzzy set go from bottom to top 
across the object image. 

30 9. The apparatus of claim 8, further comprising: 
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the at least one segment of the current frame is a bounding box. 

1 0. The apparatus of claim 9. further comprising: 

the columns associated with each of the horizontal fuzzy sets are 
overlapping into respective adjacent horizontal fuzzy sets; and, 

the rows associated with each of the vertical fuzzy sets are overlapping into 
respective adjacent vertical fuzzy sets. 

1 1 . The apparatus of claim 10, further comprising: 

the feature set computation apparatus further comprising a fuzzy set edge 
value score computing apparatus adapted to compute a fuzzy set edge value based 
upon the respective horizontal edge value or vertical edge value for the respective 
column or row and the respective horizontal fuzzy set weighting function value for 
the respective column or the respective vertical fuzzy set weighting function for the 
respective row. 

12. A video detection apparatus for detecting and following movement of a 
predefined object of interest within the video data of the output of a video camera 
encoded in frame-wise data, each frame occurring at sequential time intervals, and 
including the segmentation of at least one portion of the frame-wise data, an object 
segment, as potentially including the preselected object of interest, and including 
vertical edge pixels and horizontal edge pixels detected to be present within the 
object segment, comprising: 

an edge symmetry detection means comprising: 

a vertical symmetry detection means for computing the vertical symmetry 
of at least one of the set of vertical edge pixels and the set of horizontal 
edge pixels in each column about a horizontal axis passing through the 
object segment, and producing a vertical symmetry value for columns 
within the object segment; 

a horizontal symmetry detection means for computing the horizontal 
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symmetry of at least one of the set of vertical edge pixels and set of 
horizontal edge pixels in each row about a vertical axis passing through the 
object segment, and producing a horizontal symmetry value for rows within 
the object segment; and. 
5 an edge symmetry feature set computation means for computing an edge 

symmetry feature set for the object segment based upon the vertical 
symmetry values and the horizontal symmetry values; and, 
a background modeling means comprising: 

a background modeling means for isolating from a current frame at least 
1 0 one segment of the current frame, each of the at least one segments 

containing a possible object of interest within the current frame, on a frame- 
wise basis, and providing as an output each of the at least one segments; 
an object of interest classifying means for determining whether or not any 
segment of the at least one segments output from the background modeling 
1 5 means includes a object of interest and for characterizing any such segment 

as an object segment, utilizing at least in part the edge symmetry feature 
set; 

an object segment tracking means for tracking the location within the 
current frame of any object segment, and determining a projected location 
2 0 of the object segment in a subsequent frame, and providing the background 

modeling means with the projected location in the subsequent frame of the 
object segment; and 

wherein the background modeling means includes means for providing as 
its output the projected location of the object segment in the subsequent 
2 5 frame in lieu of a segment from the subsequent frame determined by the 

background modeling means to contain a possible object of interest or in 
the event that the background modeling means is unable in the subsequent 
frame to locate any segment containing a possible object of interest. 

30 13. The apparatus of claim 12, further comprising: 
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the edge symmetry unit further comprising: 

the vertical symmetry detection means is a means for computing the vertical 
symmetry of the set of vertical edge pixels and the set of horizontal edge 
pixels in each column about a horizontal axis passing through the object 
5 segment, and producing a vertical symmetry value for columns within the 

object segment; 

the horizontal symmetry detection means is a means for computing the 
horizontal symmetry of the set of vertical edge pixels and the set of 
horizontal edge pixels in each row about a vertical axis passing through the 
1 0 object segment, and producing a horizontal symmetry value for rows within 

the object segment. 



14. The apparatus of claim 13, further comprising: 

the object segment tracking means further comprising: 
15 a matching means for matching the location of an object segment in the 

current frame to one of a plurality of projected locations of the object 
segment in the current frame, which projections are based upon the location 
of the respective object segment in at least one prior frame; and, 
a track providing means for receiving and storing the location of an object 
2 0 segment in the current frame and over at least one of a plurality of prior 

frames and for providing the projections of the location of the object 
segment in a subsequent frame based upon the locations of the object 
segments in the current frame and at least one of the plurality of prior 
frames. 

25 

1 5. The apparatus of claim 1 4, further comprising: 

the vertical symmetry detection means includes means for computing the 
vertical symmetry value of the set of horizontal edges and the set of vertical edges 
in a given column / as the ratio of the number of horizontal edge pixels or vertical 
30 edge pixels, respectively, in each column / on opposing sides of the horizontal axis, 
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independent of which side has more horizontal edge pixels or vertical edge pixels, 
respectively, expressed as a number between 0, if there are no horizontal edge 
pixels or no vertical edge pixels, respectively, on either side of the horizontal axis 
and 1 .0 if there are, respectively, an equal number of horizontal edge pixels or 
5 vertical edge pixels, respectively on both sides of the horizontal axis; 

the horizontal symmetry detection means includes means for computing the 
horizontal symmetry value of the set of horizontal edges and the set of vertical 
edges in a given row j as the ratio of the number of horizontal edge pixels or 
vertical edge pixels, respectively, in each row j on opposing sides of the vertical 
10 axis, independent of which side has more horizontal edge pixels or vertical edge 
pixels, respectively, expressed as a number between 0, if there are no horizontal 
edge pixels or no vertical edge pixels, respectively, on either side of the horizontal 
axis and 1 .0 if there are, respectively, an equal number of horizontal edge pixels or 
vertical edge pixels, respectively on both sides of the vertical axis. 

15 

16. The apparatus of claim 1 5, further comprising: 

the background modeling means including means for providing as its 
output the projected location of the object segment in the subsequent frame in order 
to override any conflict in the labeling of pixels in the subsequent frame as part of a 
2 0 segment containing a possible object of interest. 

I 7. The apparatus of claim 16, further comprising: 

the feature set computation means further comprising: 

a horizontal fuzzy set computation means for computing a fuzzy set value 

2 5 for each of a plurality of horizontal fuzzy sets each having a selected 

weighting function applicable to a selected group of columns within the 
object segment, with the columns grouped according to position along the 
horizontal axis in the object segment; and 

a vertical fuzzy set computation means for computing a fuzzy set value for 

3 0 each of a plurality of vertical fuzzy sets each having a selected weighting 
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function applicable to a selected group of rows within the object segment, 
with the rows grouped according to position along the vertical axis in the 
object segment. 

1 8. The apparatus of claim 1 7, further comprising: 

the background modeling means including means for isolating from a 
current frame the at least one segment of the current frame utilizing the video 
object plane extracted from the current frame. 

1 9. The apparatus of claim 1 8, further comprising: 

the horizontal fuzzy sets further comprising: 

a left extreme fuzzy set having a weighting function that is a constant value 
for a first selected number of columns from the left border of the object 
segment toward the vertical axis of the object segment and decreases to 
zero over a second selected number of columns toward the vertical axis if 
the object segment; 

a right extreme fuzzy set having a weighting function that is a constant 
value for a third selected number of columns from the right border of the 
object segment toward the vertical axis of the object segment and decreases 
to zero over a fourth selected number of columns toward the vertical axis of 
the object segment; and 

at least one horizontal internal fuzzy set having a weighting function that 
increases from zero to a peak value and decreases to zero as the columns 
contained in each such internal fuzzy set go from left to right across the 
object image; and, 

the vertical fuzzy sets further comprising: 

a bottom extreme fuzzy set having a weighting function that is a constant 
value for a fifth selected number of rows from the bottom border of the 
object segment toward the horizontal axis of the object segment and 
decreases to zero over a sixth selected number of rows toward the 
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horizontal axis if the object segment; 

a top extreme fuzzy set having a weighting function that is a constant value 
for a seventh selected number of columns from the top border of the object 
segment toward the horizontal axis of the object segment and decreases to 
zero over an eighth selected number of rows toward the horizontal axis of 
the object segment; and. 

at least one vertical internal fuzzy set having a weighting function that 
increases from zero to a peak value and decreases to zero as the rows 
contained in each such vertical internal fuzzy set go from bottom to top 
across the object image. 

20. The apparatus of claim 19. further comprising: 

the at least one segment of the current frame isolated from the current frame 
by the background modeling means is a bounding box. 

2 1 . The apparatus of claim 20, further comprising: 

the columns associated with each of the horizontal fuzzy sets are 
overlapping into respective adjacent horizontal fuzzy sets; and, 

the rows associated with each of the vertical fuzzy sets are overlapping into 
respective adjacent vertical fuzzy sets. 

22. The apparatus of claim 21, further comprising: 

the feature set computation means further comprising a fuzzy set edge value 
score computing means for computing a fuzzy set edge value based upon the 
respective horizontal edge value or vertical edge value for the respective column or 
row and the respective horizontal fuzzy set weighting function value for the 
respective column or the respective vertical fuzzy set weighting function for the 
respective row. 

23. A video detection method for detecting and following movement of a 
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predefined object of interest within the video data of the output of a video camera 
encoded in frame-wise data, each frame occurring at sequential time intervals, and 
including the segmentation of at least one portion of the frame-wise data, an object 
segment, as potentially including the preselected object of interest, and including 
vertical edge pixels and horizontal edge pixels detected to be present within the 
object segment, comprising: 

utilizing an edge symmetry detection unit: 

computing the vertical symmetry of at least one of the set of vertical edge 
pixels and the set of horizontal edge pixels in each column about a 
horizontal axis passing through the object segment, and producing a vertical 
symmetry value for columns within the object segment; 
computing the horizontal symmetry of at least one of the set of vertical edge 
pixels and set of horizontal edge pixels in each row about a vertical axis 
passing through the object segment, and producing a horizontal symmetry 
value for rows within the object segment; and, 

computing an edge symmetry feature set for the object segment based upon 
the vertical symmetry values and the horizontal symmetry values; and, 
utilizing a background modeling unit comprising: 
isolating from a current frame at least one segment of the current frame, 
each of the at least one segments containing a possible object of interest 
within the current frame, on a frame-wise basis, and providing as an output 
each of the at least one segments; 

determining whether or not any segment of the at least one segments output 
from the background modeling means includes a object of interest and for 
characterizing any such segment as an object segment, utilizing at least in 
part the edge symmetry feature set; 

tracking the location within the current frame of any object segment, and 
determining a projected location of the object segment in a subsequent 
frame, and providing the background modeling means with the projected 
location in the subsequent frame of the object segment; and 
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wherein the background modeling unit includes means for providing as its 
output the projected location of the object segment in the subsequent frame 
in lieu of a segment from the subsequent frame determined by the 
background modeling means to contain a possible object of interest or in 
the event that the background modeling means is unable in the subsequent 
frame to locate any segment containing a possible object of interest. 

24. The method of claim 23, further comprising: 

the utilization of the edge symmetry unit further comprising: 
computing the vertical symmetry of the set of vertical edge pixels and the 
set of horizontal edge pixels in each column about a horizontal axis passing 
through the object segment, and producing a vertical symmetry value for 
columns within the object segment; and, 

computing the horizontal symmetry of the set of vertical edge pixels and the 
set of horizontal edge pixels in each row about a vertical axis passing 
through the object segment, and producing a horizontal symmetry value for 
rows within the object segment. 

25. The method of claim 24, further comprising: 

the utilization of the object segment tracking means further comprising: 
matching the location of an object segment in the current frame to one of a 
plurality of projected locations of the object segment in the current frame, 
which projections are based upon the location of the respective object 
segment in at least one prior frame; and, 

receiving and storing the location of an object segment in the current frame 
and over at least one of a plurality of prior frames and for providing the 
projections of the location of the object segment in a subsequent frame 
based upon the locations of the object segments in the current frame and at 
least one of the plurality of prior frames. 



-60- 

c: Bill \pa tent applications draf ted\XLaboratories\1044-4 13-01 .doc 



26. The method of claim 25, further comprising: 

computing the vertical symmetry value of the set of horizontal edges and 
the set of vertical edges in a given column / as the ratio of the number of horizontal 
edge pixels or vertical edge pixels, respectively, in each column / on opposing 
5 sides of the horizontal axis, independent of which side has more horizontal edge 
pixels or vertical edge pixels, respectively, expressed as a number between 0. if 
there are no horizontal edge pixels or no vertical edge pixels, respectively, on 
either side of the horizontal axis and 1 .0 if there are, respectively, an equal number 
of horizontal edge pixels or vertical edge pixels, respectively on both sides of the 

1 0 horizontal axis; and, 

computing the horizontal symmetry value of the set of horizontal edges and 
the set of vertical edges in a given row / as the ratio of the number of horizontal 
edge pixels or vertical edge pixels, respectively, in each row; on opposing sides of 
the vertical axis, independent of which side has more horizontal edge pixels or 

15 vertical edge pixels, respectively, expressed as a number between 0, if there are no 
horizontal edge pixels or no vertical edge pixels, respectively, on either side of the 
horizontal axis and 1 .0 if there are, respectively, an equal number of horizontal 
edge pixels or vertical edge pixels, respectively on both sides of the vertical axis. 

2 0 27. The method of claim 26, further comprising: 

the background modeling means including means for providing as its 
output the projected location of the object segment in the subsequent frame in order 
to override any conflict in the labeling of pixels in the subsequent frame as part of a 
segment containing a possible object of interest. 

25 

28. The method of claim 27, further comprising: 

computing a fuzzy set value for each of a plurality of horizontal fuzzy sets 
each having a selected weighting function applicable to a selected group of 
columns within the object segment, with the columns grouped according to 
30 position along the horizontal axis in the object segment; and 
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computing a fuzzy set value for each of a plurality of vertical fuzzy sets 
each having a selected weighting function applicable to a selected group of rows 
within the object segment, with the rows grouped according to position along the 
vertical axis in the object segment. 

29. The apparatus of claim 28. further comprising: 

isolating from a current frame the at least one segment of the current frame 
utilizing the video object plane extracted from the current frame. 



1 0 30. The method of claim 29, further comprising: 

the horizontal fuzzy sets further comprising: 

a left extreme fuzzy set having a weighting function that is a constant value 
for a first selected number of columns from the left border of the object 
segment toward the vertical axis of the object segment and decreases to 
1 5 zero over a second selected number of columns toward the vertical axis if 

the object segment; 

a right extreme fuzzy set having a weighting function that is a constant 
value for a third selected number of columns from the right border of the 
object segment toward the vertical axis of the object segment and decreases 
2 0 to zero over a fourth selected number of columns toward the vertical axis of 

the object segment; and 

at least one horizontal internal fuzzy set having a weighting function that 
increases from zero to a peak value and decreases to zero as the columns 
contained in each such internal fuzzy set go from left to right across the 
2 5 object image; and, 

the vertical fuzzy sets further comprising: 

a bottom extreme fuzzy set having a weighting function that is a constant 
value for a fifth selected number of rows from the bottom border of the 
object segment toward the horizontal axis of the object segment and 
30 decreases to zero over a sixth selected number of rows toward the 
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horizontal axis if the object segment; 

a top extreme fuzzy set having a weighting function that is a constant value 
for a seventh selected number of columns from the top border of the object 
segment toward the horizontal axis of the object segment and decreases to 
5 zero over an eighth selected number of rows toward the horizontal axis of 

the object segment; and, 

at least one vertical internal fuzzy set having a weighting function that 
increases from zero to a peak value and decreases to zero as the rows 
contained in each such vertical internal fuzzy set go from bottom to top 
1 0 across the object image. 

3 1 . The apparatus of claim 30, further comprising: 

the at least one segment of the current frame isolated from the current frame 
by the background modeling means is a bounding box. 

15 

32. The method of claim 3 1 , further comprising: 

the columns associated with each of the horizontal fuzzy sets are 
overlapping into respective adjacent horizontal fuzzy sets; and, 

the rows associated with each of the vertical fuzzy sets are overlapping into 
2 0 respective adjacent vertical fuzzy sets. 

33. The method of claim 32, further comprising: 

computing a fuzzy set edge value based upon the respective horizontal edge 
value or vertical edge value for the respective column or row and the respective 

2 5 horizontal fuzzy set weighting function value for the respective column or the 

respective vertical fuzzy set weighting function for the respective row. 

34. A video detection apparatus adapted to detect and follow movement of a 
predefined object of interest within the video data of the output of a video camera 

3 0 encoded in frame-wise data, each frame occurring at sequential time intervals, and 
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including the segmentation of at least one portion of the frame-wise data, an object 
segment, as potentially including the preselected object of interest, and including 
vertical edge pixels and horizontal edge pixels detected to be present within the 
object segment, comprising: 

an edge symmetry detection unit comprising: 

a vertical symmetry detector adapted to compute the vertical symmetry of 
the set of vertical edge pixels and the set of horizontal edge pixels in each 
column about a horizontal axis passing through the object segment, and to 
produce a vertical symmetry value for columns within the object segment; 
a horizontal symmetry detector adapted to compute the horizontal 
symmetry of the set of vertical edge pixels and set of horizontal edge pixels 
in each row about a vertical axis passing through the object segment, and to 
produce a horizontal symmetry value for rows within the object segment; 
and, 

an edge symmetry feature set computation apparatus adapted to compute an 
edge symmetry feature set for the object segment based upon the vertical 
symmetry values and the horizontal symmetry values; and, 
a background modeling unit comprising: 

a background modeling apparatus adapted to isolate from a current frame at 
least one segment of the current frame, each of the at least one segments 
containing a possible object of interest within the current frame, on a frame- 
wise basis, and to provide as an output each of the at least one segments; 
an object of interest classifying apparatus adapted to determine whether or 
not any segment of the at least one segments output from the background 
modeling apparatus includes a object of interest and to characterize any 
such segment as an object segment, utilizing at least in part the edge 
symmetry feature set; 

an object segment tracking apparatus adapted to track the location within 
the current frame of any object segment, and to determine a projected 
location of the object segment in a subsequent frame, and to provide the 
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background modeling apparatus with the projected location in the 
subsequent frame of the object segment; 

wherein the background modeling apparatus is adapted to provide as its 
output the projected location of the object segment in the subsequent frame 
in lieu of a segment from the subsequent frame determined by the 
background modeling apparatus to contain a possible object of interest or in 
the event that the background modeling apparatus is unable in the 
subsequent frame to locate any segment containing a possible object of 
interest; and, 

the object segment tracking apparatus further comprising: 

a matching apparatus adapted to match the location of an object segment in 

the current frame to one of a plurality of projected locations of the object 

segment in the current frame, which projections are based upon the location 

of the respective object segment in at least one prior frame; and, 

a track provider adapted to receive and store the location of an object 

segment in the current frame and over a plurality of prior frames and 

adapted to provide the projections of the location of the object segment in a 

subsequent frame based upon the locations of the object segments in the 

current frame and the plurality of prior frames. 

35. A video detection apparatus adapted to detect and follow movement of a 
predefined object of interest within the video data of the output of a video camera 
encoded in frame-wise data, each frame occurring at sequential time intervals, and 
including the segmentation of at least one portion of the frame-wise data, an object 
segment, as potentially including the preselected object of interest, and including 
vertical edge pixels and horizontal edge pixels detected to be present within the 
object segment, comprising: 

an edge symmetry detection unit comprising: 

a vertical symmetry detector adapted to compute the vertical symmetry of 
the set of vertical edge pixels and the set of horizontal edge pixels in each 
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column about a horizontal axis passing through the object segment, and to 
produce a vertical symmetry value for columns within the object segment; 
a horizontal symmetry detector adapted to compute the horizontal 
symmetry of the set of vertical edge pixels and set of horizontal edge pixels 
in each row about a vertical axis passing through the object segment, and to 
produce a horizontal symmetry value for rows within the object segment; 
and, 

an edge symmetry feature set computation apparatus adapted to compute an 
edge symmetry feature set for the object segment based upon the vertical 
symmetry values and the horizontal symmetry values; and, 
a background modeling unit comprising: 

a background modeling apparatus adapted to isolate from a current frame at 
least one segment of the current frame, each of the at least one segments 
containing a possible object of interest within the current frame, on a frame- 
wise basis, and to provide as an output each of the at least one segments; 
an object of interest classifying apparatus adapted to determine whether or 
not any segment of the at least one segments output from the background 
modeling apparatus includes a object of interest and to characterize any 
such segment as an object segment, utilizing at least in part the edge 
symmetry feature set; 

an object segment tracking apparatus adapted to track the location within 
the current frame of any object segment, and to determine a projected 
location of the object segment in a subsequent frame, and to provide the 
background modeling apparatus with the projected location in the 
subsequent frame of the object segment; 

wherein the background modeling apparatus is adapted to provide as its 
output the projected location of the object segment in the subsequent frame 
in lieu of a segment from the subsequent frame determined by the 
background modeling apparatus to contain a possible object of interest or in 
the event that the background modeling apparatus is unable in the 
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subsequent frame to locate any segment containing a possible object of 
interest; and, 

the object segment tracking apparatus further comprising: 
a matching apparatus adapted to match the location of an object segment in 
5 the current frame to one of a plurality of projected locations of the object 

segment in the current frame, which projections are based upon the location 
of the respective object segment in at least one prior frame; 

a track provider adapted to receive and store the location of an object segment in 

the current frame and over a plurality of prior frames and adapted to provide the 
1 0 projections of the location of the object segment in a subsequent frame based upon 

the locations of the object segments in the current frame and the plurality of prior 

frames; and, 

the vertical symmetry detector is adapted to compute the vertical symmetry 
value of at least one of the set of horizontal edges or the set of vertical edges in a 

15 given column / as the ratio of the number of horizontal edge pixels or vertical edge 
pixels, respectively, in each column / on opposing sides of the horizontal axis, 
independent of which side has more horizontal edge pixels or vertical edge pixels, 
respectively, expressed as a number between 0, if there are no horizontal edge 
pixels or no vertical edge pixels, respectively, on either side of the horizontal axis 

2 0 and 1 .0 if there are, respectively, an equal number of horizontal edge pixels or 
vertical edge pixels, respectively on both sides of the horizontal axis; and, 

the horizontal symmetry detector is adapted to compute the horizontal 
symmetry value of at least one of the set of horizontal edges or the set of vertical 
edges in a given row j as the ratio of the number of horizontal edge pixels or 

2 5 vertical edge pixels, respectively, in each row j on opposing sides of the vertical 
axis, independent of which side has more horizontal edge pixels or vertical edge 
pixels, respectively, expressed as a number between 0, if there are no horizontal 
edge pixels or no vertical edge pixels, respectively, on either side of the horizontal 
axis and 1 .0 if there are, respectively, an equal number of horizontal edge pixels or 

30 vertical edge pixels, respectively on both sides of the vertical axis. 
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36. A video detection apparatus adapted to detect and follow movement of a 
predefined object of interest within the video data of the output of a video camera 
encoded in frame-wise data, each frame occurring at sequential time intervals, and 
including the segmentation of at least one portion of the frame-wise data, an object 
segment, as potentially including the preselected object of interest, and including 
vertical edge pixels and horizontal edge pixels detected to be present within the 
object segment, comprising: 

an edge symmetry detection unit comprising: 

a vertical symmetry detector adapted to compute the vertical symmetry of 
the set of vertical edge pixels and the set of horizontal edge pixels in each 
column about a horizontal axis passing through the object segment, and to 
produce a vertical symmetry value for columns within the object segment; 
a horizontal symmetry detector adapted to compute the horizontal 
symmetry of the set of vertical edge pixels and set of horizontal edge pixels 
in each row about a vertical axis passing through the object segment, and to 
produce a horizontal symmetry value for rows within the object segment; 
and, 

an edge symmetry feature set computation apparatus adapted to compute an 
edge symmetry feature set for the object segment based upon the vertical 
symmetry values and the horizontal symmetry values; and, 
a background modeling unit comprising: 

a background modeling apparatus adapted to isolate from a current frame at 
least one segment of the current frame, each of the at least one segments 
containing a possible object of interest within the current frame, on a frame- 
wise basis, and to provide as an output each of the at least one segments; 
an object of interest classifying apparatus adapted to determine whether or 
not any segment of the at least one segments output from the background 
modeling apparatus includes a object of interest and to characterize any 
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such segment as an object segment, utilizing at least in part the edge 
symmetry feature set; 

an object segment tracking apparatus adapted to track the location within 
the current frame of any object segment and to determine a projected 
location of the object segment in a subsequent frame, and to provide the 
background modeling apparatus with the projected location in the 
subsequent frame of the object segment; 

wherein the background modeling apparatus is adapted to provide as its 
output the projected location of the object segment in the subsequent frame 
in lieu of a segment from the subsequent frame determined by the 
background modeling apparatus to contain a possible object of interest or in 
the event that the background modeling apparatus is unable in the 
subsequent frame to locate any segment containing a possible object of 
interest; and, 

the object segment tracking apparatus further comprising: 
a matching apparatus adapted to match the location of an object segment in 
the current frame to one of a plurality of projected locations of the object 
segment in the current frame, which projections are based upon the location 
of the respective object segment in at least one prior frame; 
a track provider adapted to receive and store the location of an object segment in 
the current frame and over a plurality of prior frames and adapted to provide the 
projections of the location of the object segment in a subsequent frame based upon 
the locations of the object segments in the current frame and the plurality of prior 
frames; and, 

the vertical symmetry detector is adapted to compute the vertical symmetry 
value of at least one of the set of horizontal edges or the set of vertical edges in a 
given column i as the ratio of the number of horizontal edge pixels or vertical edge 
pixels, respectively, in each column / on opposing sides of the horizontal axis, 
independent of which side has more horizontal edge pixels or vertical edge pixels, 
respectively, expressed as a number between 0, if there are no horizontal edge 
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pixels or no vertical edge pixels, respectively, on either side of the horizontal axis 
and 1 .0 if there are, respectively, an equal number of horizontal edge pixels or 
vertical edge pixels, respectively on both sides of the horizontal axis; and, 

the horizontal symmetry detector is adapted to compute the horizontal 
5 symmetry value of at least one of the set of horizontal edges or the set of vertical 
edges in a given row j as the ratio of the number of horizontal edge pixels or 
vertical edge pixels, respectively, in each row j on opposing sides of the vertical 
axis, independent of which side has more horizontal edge pixels or vertical edge 
pixels, respectively, expressed as a number between 0, if there are no horizontal 

1 0 edge pixels or no vertical edge pixels, respectively, on either side of the horizontal 
axis and 1 .0 if there are, respectively, an equal number of horizontal edge pixels or 
vertical edge pixels, respectively on both sides of the vertical axis; and, 

the background modeling apparatus is adapted to provide as its output the 
projected location of the object segment in the subsequent frame in order to 

1 5 override any conflict in the labeling of pixels in the subsequent frame as part of a 
segment containing a possible object of interest. 

37. A video detection apparatus adapted to detect and follow movement of a 
predefined object of interest within the video data of the output of a video camera 
2 0 encoded in frame-wise data, each frame occurring at sequential time intervals, and 
including the segmentation of at least one portion of the frame-wise data, an object 
segment, as potentially including the preselected object of interest, and including 
vertical edge pixels and horizontal edge pixels detected to be present within the 
object segment, comprising: 
2 5 an edge symmetry detection unit comprising: 

a vertical symmetry detector adapted to compute the vertical symmetry of 
the set of vertical edge pixels and the set of horizontal edge pixels in each 
column about a horizontal axis passing through the object segment, and to 
produce a vertical symmetry value for columns within the object segment; 
30 a horizontal symmetry detector adapted to compute the horizontal 
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symmetry of the set of vertical edge pixels and set of horizontal edge pixels 
in each row about a vertical axis passing through the object segment, and to 
produce a horizontal symmetry value for rows within the object segment; 
and, 

5 an edge symmetry feature set computation apparatus adapted to compute an 

edge symmetry feature set for the object segment based upon the vertical 
symmetry values and the horizontal symmetry values; and, 
a background modeling unit comprising: 

a background modeling apparatus adapted to isolate from a current frame at 
1 0 least one segment of the current frame, each of the at least one segments 

containing a possible object of interest within the current frame, on a frame- 
wise basis, and to provide as an output each of the at least one segments; 
an object of interest classifying apparatus adapted to determine whether or 
not any segment of the at least one segments output from the background 
1 5 modeling apparatus includes a object of interest and to characterize any 

such segment as an object segment, utilizing at least in part the edge 
symmetry feature set; 

an object segment tracking apparatus adapted to track the location within 
the current frame of any object segment, and to determine a projected 
2 0 location of the object segment in a subsequent frame, and to provide the 

background modeling apparatus with the projected location in the 
subsequent frame of the object segment; 

wherein the background modeling apparatus is adapted to provide as its 
output the projected location of the object segment in the subsequent frame 

2 5 in lieu of a segment from the subsequent frame determined by the 

background modeling apparatus to contain a possible object of interest or in 
the event that the background modeling apparatus is unable in the 
subsequent frame to locate any segment containing a possible object of 
interest: and, 

3 0 the object segment tracking apparatus further comprising: 
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a matching apparatus adapted to match the location of an object segment in 
the current frame to one of a plurality of projected locations of the object 
segment in the current frame, which projections are based upon the location 
of the respective object segment in at least one prior frame; 
5 a track provider adapted to receive and store the location of an object segment in 
the current frame and over a plurality of prior frames and adapted to provide the 
projections of the location of the object segment in a subsequent frame based upon 
the locations of the object segments in the current frame and the plurality of prior 
frames; and, 

10 the vertical symmetry detector is adapted to compute the vertical symmetry 

value of at least one of the set of horizontal edges or the set of vertical edges in a 
given column / as the ratio of the number of horizontal edge pixels or vertical edge 
pixels, respectively, in each column /"on opposing sides of the horizontal axis, 
independent of which side has more horizontal edge pixels or vertical edge pixels, 

1 5 respectively, expressed as a number between 0, if there are no horizontal edge 

pixels or no vertical edge pixels, respectively, on either side of the horizontal axis 
and 1 .0 if there are, respectively, an equal number of horizontal edge pixels or 
vertical edge pixels, respectively on both sides of the horizontal axis; and, 

the horizontal symmetry detector is adapted to compute the horizontal 

2 0 symmetry value of at least one of the set of horizontal edges or the set of vertical 
edges in a given row ; as the ratio of the number of horizontal edge pixels or 
vertical edge pixels, respectively, in each row j on opposing sides of the vertical 
axis, independent of which side has more horizontal edge pixels or vertical edge 
pixels, respectively, expressed as a number between 0, if there are no horizontal 

2 5 edge pixels or no vertical edge pixels, respectively, on either side of the horizontal 
axis and 1 .0 if there are, respectively, an equal number of horizontal edge pixels or 
vertical edge pixels, respectively on both sides of the vertical axis; 

the background modeling apparatus is adapted to provide as its output the 
projected location of the object segment in the subsequent frame in order to 

30 override any conflict in the labeling of pixels in the subsequent frame as part of a 
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segment containing a possible object of interest; and, 

the feature set computation apparatus further comprising: 
a horizontal fuzzy set computation apparatus adapted to compute a fuzzy 
set value for each of a plurality of horizontal fuzzy sets each having a 
5 selected weighting function applicable to a selected group of columns 

within the object segment, with the columns grouped according to position 
along the horizontal axis in the object segment; and, 
a vertical fuzzy set computation apparatus adapted to compute a fuzzy set 
value for each of a plurality of vertical fuzzy sets each having a selected 
1 0 weighting function applicable to a selected group of rows within the object 

segment, with the rows grouped according to position along the vertical 
axis in the object segment. 

38. A video detection apparatus adapted to detect and follow movement of a 
1 5 predefined object of interest within the video data of the output of a video camera 
encoded in frame-wise data, each frame occurring at sequential time intervals, and 
including the segmentation of at least one portion of the frame-wise data, an object 
segment, as potentially including the preselected object of interest, and including 
vertical edge pixels and horizontal edge pixels detected to be present within the 
2 0 object segment, comprising: 

an edge symmetry detection unit comprising: 

a vertical symmetry detector adapted to compute the vertical symmetry of 
the set of vertical edge pixels and the set of horizontal edge pixels in each 
column about a horizontal axis passing through the object segment, and to 

2 5 produce a vertical symmetry value for columns within the object segment; 

a horizontal symmetry detector adapted to compute the horizontal 
symmetry of the set of vertical edge pixels and set of horizontal edge pixels 
in each row about a vertical axis passing through the object segment, and to 
produce a horizontal symmetry value for rows within the object segment; 

30 and, 
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an edge symmetry feature set computation apparatus adapted to compute an 
edge symmetry feature set for the object segment based upon the vertical 
symmetry values and the horizontal symmetry values; and, 
a background modeling unit comprising: 
5 a background modeling apparatus adapted to isolate from a current frame at 

least one segment of the current frame, each of the at least one segments 
containing a possible object of interest within the current frame, on a frame- 
wise basis, and to provide as an output each of the at least one segments; 
an object of interest classifying apparatus adapted to determine whether or 
1 0 not any segment of the at least one segments output from the background 

modeling apparatus includes a object of interest and to characterize any 
such segment as an object segment, utilizing at least in part the edge 
symmetry feature set; 

an object segment tracking apparatus adapted to track the location within 
1 5 the current frame of any object segment, and to determine a projected 

location of the object segment in a subsequent frame, and to provide the 
background modeling apparatus with the projected location in the 
subsequent frame of the object segment; 

wherein the background modeling apparatus is adapted to provide as its 
2 0 output the projected location of the object segment in the subsequent frame 

in lieu of a segment from the subsequent frame determined by the 
background modeling apparatus to contain a possible object of interest or in 
the event that the background modeling apparatus is unable in the 
subsequent frame to locate any segment containing a possible object of 
2 5 interest; and, 

the object segment tracking apparatus further comprising: 
a matching apparatus adapted to match the location of an object segment in 
the current frame to one of a plurality of projected locations of the object 
segment in the current frame, which projections are based upon the location 
30 of the respective object segment in at least one prior frame; 
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a track provider adapted to receive and store the location of an object segment in 
the current frame and over a plurality of prior frames and adapted to provide the 
projections of the location of the object segment in a subsequent frame based upon 
the locations of the object segments in the current frame and the plurality of prior 
5 frames; and, 

the vertical symmetry detector is adapted to compute the vertical symmetry 
value of at least one of the set of horizontal edges or the set of vertical edges in a 
given column / as the ratio of the number of horizontal edge pixels or vertical edge 
pixels, respectively, in each column / on opposing sides of the horizontal axis, 

10 independent of which side has more horizontal edge pixels or vertical edge pixels, 
respectively, expressed as a number between 0, if there are no horizontal edge 
pixels or no vertical edge pixels, respectively, on either side of the horizontal axis 
and 1 .0 if there are, respectively, an equal number of horizontal edge pixels or 
vertical edge pixels, respectively on both sides of the horizontal axis; and, 

1 5 the horizontal symmetry detector is adapted to compute the horizontal 

symmetry value of at least one of the set of horizontal edges or the set of vertical 
edges in a given row j as the ratio of the number of horizontal edge pixels or 
vertical edge pixels, respectively, in each row j on opposing sides of the vertical 
axis, independent of which side has more horizontal edge pixels or vertical edge 

2 0 pixels, respectively, expressed as a number between 0, if there are no horizontal 
edge pixels or no vertical edge pixels, respectively, on either side of the horizontal 
axis and 1 .0 if there are, respectively, an equal number of horizontal edge pixels or 
vertical edge pixels, respectively on both sides of the vertical axis; 

the background modeling apparatus is adapted to provide as its output the 

2 5 projected location of the object segment in the subsequent frame in order to 

override any conflict in the labeling of pixels in the subsequent frame as part of a 
segment containing a possible object of interest; 

the feature set computation apparatus further comprising: 

a horizontal fuzzy set computation apparatus adapted to compute a fuzzy 

30 set value for each of a plurality of horizontal fuzzy sets each having a 
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selected weighting function applicable to a selected group of columns 

within the object segment, with the columns grouped according to position 

along the horizontal axis in the object segment; and, 
a vertical fuzzy set computation apparatus adapted to compute a fuzzy set value for 
5 each of a plurality of vertical fuzzy sets each having a selected weighting function 
applicable to a selected group of rows within the object segment, with the rows 
grouped according to position along the vertical axis in the object segment; and, 

the background modeling apparatus is adapted to isolate from a current 
frame the at least one segment of the current frame utilizing the video object plane 
1 0 extracted from the current frame. 



39. A video detection apparatus adapted to detect and follow movement of a 
predefined object of interest within the video data of the output of a video camera 
encoded in frame-wise data, each frame occurring at sequential time intervals, and 
15 including the segmentation of at least one portion of the frame-wise data, an object 
segment, as potentially including the preselected object of interest, and including 
vertical edge pixels and horizontal edge pixels detected to be present within the 
object segment, comprising: 

an edge symmetry detection unit comprising: 
2 0 a vertical symmetry detector adapted to compute the vertical symmetry of 

the set of vertical edge pixels and the set of horizontal edge pixels in each 
column about a horizontal axis passing through the object segment, and to 
produce a vertical symmetry value for columns within the object segment; 
a horizontal symmetry detector adapted to compute the horizontal 
2 5 symmetry of the set of vertical edge pixels and set of horizontal edge pixels 

in each row about a vertical axis passing through the object segment, and to 
produce a horizontal symmetry value for rows within the object segment; 
and, 

an edge symmetry feature set computation apparatus adapted to compute an 
30 edge symmetry feature set for the object segment based upon the vertical 
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symmetry values and the horizontal symmetry values; and, 
a background modeling unit comprising: 

a background modeling apparatus adapted to isolate from a current frame at 
least one segment of the current frame, each of the at least one segments 
containing a possible object of interest within the current frame, on a frame- 
wise basis, and to provide as an output each of the at least one segments; 
an object of interest classifying apparatus adapted to determine whether or 
not any segment of the at least one segments output from the background 
modeling apparatus includes a object of interest and to characterize any 
such segment as an object segment, utilizing at least in part the edge 
symmetry feature set; 

an object segment tracking apparatus adapted to track the location within 
the current frame of any object segment, and to determine a projected 
location of the object segment in a subsequent frame, and to provide the 
background modeling apparatus with the projected location in the 
subsequent frame of the object segment; 

wherein the background modeling apparatus is adapted to provide as its 
output the projected location of the object segment in the subsequent frame 
in lieu of a segment from the subsequent frame determined by the 
background modeling apparatus to contain a possible object of interest or in 
the event that the background modeling apparatus is unable in the 
subsequent frame to locate any segment containing a possible object of 
interest; and, 

the object segment tracking apparatus further comprising: 
a matching apparatus adapted to match the location of an object segment in 
the current frame to one of a plurality of projected locations of the object 
segment in the current frame, which projections are based upon the location 
of the respective object segment in at least one prior frame; 
a track provider adapted to receive and store the location of an object segment in 
the current frame and over a plurality of prior frames and adapted to provide the 
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projections of the location of the object segment in a subsequent frame based upon 
the locations of the object segments in the current frame and the plurality of prior 
frames; and. 

the vertical symmetry detector is adapted to compute the vertical symmetry 
5 value of at least one of the set of horizontal edges or the set of vertical edges in a 
given column /' as the ratio of the number of horizontal edge pixels or vertical edge 
pixels, respectively, in each column / on opposing sides of the horizontal axis, 
independent of which side has more horizontal edge pixels or vertical edge pixels, 
respectively, expressed as a number between 0, if there are no horizontal edge 
10 pixels or no vertical edge pixels, respectively, on either side of the horizontal axis 
and 1 .0 if there are, respectively, an equal number of horizontal edge pixels or 
vertical edge pixels, respectively on both sides of the horizontal axis; and, 

the horizontal symmetry detector is adapted to compute the horizontal 
symmetry value of at least one of the set of horizontal edges or the set of vertical 
1 5 edges in a given row j as the ratio of the number of horizontal edge pixels or 

vertical edge pixels, respectively, in each row j on opposing sides of the vertical 
axis, independent of which side has more horizontal edge pixels or vertical edge 
pixels, respectively, expressed as a number between 0, if there are no horizontal 
edge pixels or no vertical edge pixels, respectively, on either side of the horizontal 
2 0 axis and 1 .0 if there are, respectively, an equal number of horizontal edge pixels or 
vertical edge pixels, respectively on both sides of the vertical axis; 

the background modeling apparatus is adapted to provide as its output the 
projected location of the object segment in the subsequent frame in order to 
override any conflict in the labeling of pixels in the subsequent frame as part of a 
2 5 segment containing a possible object of interest; 

the feature set computation apparatus further comprising: 
a horizontal fuzzy set computation apparatus adapted to compute a fuzzy 
set value for each of a plurality of horizontal fuzzy sets each having a 
selected weighting function applicable to a selected group of columns 
30 within the object segment, with the columns grouped according to position 
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along the horizontal axis in the object segment; and, 
a vertical fuzzy set computation apparatus adapted to compute a fuzzy set value for 
each of a plurality of vertical fuzzy sets each having a selected weighting function 
applicable to a selected group of rows within the object segment, with the rows 
grouped according to position along the vertical axis in the object segment; and, 

the background modeling apparatus is adapted to isolate from a current 
frame the at least one segment of the current frame utilizing the video object plane 
extracted from the current frame; and 

the horizontal fuzzy sets further comprising: 

a left extreme fuzzy set having a weighting function that is a constant value 
for a first selected number of columns from the left border of the object 
segment toward the vertical axis of the object segment and decreases to 
zero over a second selected number of columns toward the vertical axis if 
the object segment; 

a right extreme fuzzy set having a weighting function that is a constant 
value for a third selected number of columns from the right border of the 
object segment toward the vertical axis of the object segment and decreases 
to zero over a fourth selected number of columns toward the vertical axis of 
the object segment; and 

at least one horizontal internal fuzzy set having a weighting function that 
increases from zero to a peak value and decreases to zero as the columns 
contained in each such internal fuzzy set go from left to right across the 
object image; 

the vertical fuzzy sets further comprising: 

a bottom extreme fuzzy set having a weighting function that is a constant 
value for a fifth selected number of rows from the bottom border of the 
object segment toward the horizontal axis of the object segment and 
decreases to zero over a sixth selected number of rows toward the 
horizontal axis if the object segment; 

a top extreme fuzzy set having a weighting function that is a constant value 
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for a seventh selected number of columns from the top border of the object 
segment toward the horizontal axis of the object segment and decreases to 
zero over an eighth selected number of rows toward the horizontal axis of 
the object segment; and, 
5 at least one vertical internal fuzzy set having a weighting function that increases 
from zero to a peak value and decreases to zero as the rows contained in each such 
vertical internal fuzzy set go from bottom to top across the object image. 

40. A video detection apparatus adapted to detect and follow movement of a 
1 0 predefined object of interest within the video data of the output of a video camera 
encoded in frame-wise data, each frame occurring at sequential time intervals, and 
including the segmentation of at least one portion of the frame-wise data, an object 
segment, as potentially including the preselected object of interest, and including 
vertical edge pixels and horizontal edge pixels detected to be present within the 
1 5 object segment, comprising: 

an edge symmetry detection unit comprising: 

a vertical symmetry detector adapted to compute the vertical symmetry of 
the set of vertical edge pixels and the set of horizontal edge pixels in each 
column about a horizontal axis passing through the object segment, and to 

2 0 produce a vertical symmetry value for columns within the object segment; 

a horizontal symmetry detector adapted to compute the horizontal 
symmetry of the set of vertical edge pixels and set of horizontal edge pixels 
in each row about a vertical axis passing through the object segment, and to 
produce a horizontal symmetry value for rows within the object segment; 

2 5 and, 

an edge symmetry feature set computation apparatus adapted to compute an 
edge symmetry feature set for the object segment based upon the vertical 
symmetry values and the horizontal symmetry values; and, 
a background modeling unit comprising: 
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a background modeling apparatus adapted to isolate from a current frame at 
least one segment of the current frame, each of the at least one segments 
containing a possible object of interest within the current frame, on a frame- 
wise basis, and to provide as an output each of the at least one segments; 
an object of interest classifying apparatus adapted to determine whether or 
not any segment of the at least one segments output from the background 
modeling apparatus includes a object of interest and to characterize any 
such segment as an object segment, utilizing at least in part the edge 
symmetry feature set; 

an object segment tracking apparatus adapted to track the location within 
the current frame of any object segment, and to determine a projected 
location of the object segment in a subsequent frame, and to provide the 
background modeling apparatus with the projected location in the 
subsequent frame of the object segment; 

wherein the background modeling apparatus is adapted to provide as its 
output the projected location of the object segment in the subsequent frame 
in lieu of a segment from the subsequent frame determined by the 
background modeling apparatus to contain a possible object of interest or in 
the event that the background modeling apparatus is unable in the 
subsequent frame to locate any segment containing a possible object of 
interest; and, 

the object segment tracking apparatus further comprising: 
a matching apparatus adapted to match the location of an object segment in 
the current frame to one of a plurality of projected locations of the object 
segment in the current frame, which projections are based upon the location 
of the respective object segment in at least one prior frame; 
a track provider adapted to receive and store the location of an object segment in 
the current frame and over a plurality of prior frames and adapted to provide the 
projections of the location of the object segment in a subsequent frame based upon 
the locations of the object segments in the current frame and the plurality of prior 
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frames; and, 

the vertical symmetry detector is adapted to compute the vertical symmetry 
value of at least one of the set of horizontal edges or the set of vertical edges in a 
given column / as the ratio of the number of horizontal edge pixels or vertical edge 
5 pixels, respectively, in each column / on opposing sides of the horizontal axis, 
independent of which side has more horizontal edge pixels or vertical edge pixels, 
respectively, expressed as a number between 0, if there are no horizontal edge 
pixels or no vertical edge pixels, respectively, on either side of the horizontal axis 
and 1 .0 if there are, respectively, an equal number of horizontal edge pixels or 
1 0 vertical edge pixels, respectively on both sides of the horizontal axis; and, 

the horizontal symmetry detector is adapted to compute the horizontal 
symmetry value of at least one of the set of horizontal edges or the set of vertical 
edges in a given row j as the ratio of the number of horizontal edge pixels or 
vertical edge pixels, respectively, in each row j on opposing sides of the vertical 
15 axis, independent of which side has more horizontal edge pixels or vertical edge 
pixels, respectively, expressed as a number between 0, if there are no horizontal 
edge pixels or no vertical edge pixels, respectively, on either side of the horizontal 
axis and 1 .0 if there are, respectively, an equal number of horizontal edge pixels or 
vertical edge pixels, respectively on both sides of the vertical axis; 
2 0 the background modeling apparatus is adapted to provide as its output the 

projected location of the object segment in the subsequent frame in order to 
override any conflict in the labeling of pixels in the subsequent frame as part of a 
segment containing a possible object of interest; 

the feature set computation apparatus further comprising: 
2 5 a horizontal fuzzy set computation apparatus adapted to compute a fuzzy 

set value for each of a plurality of horizontal fuzzy sets each having a 
selected weighting function applicable to a selected group of columns 
within the object segment, with the columns grouped according to position 
along the horizontal axis in the object segment; and, 
30 a vertical fuzzy set computation apparatus adapted to compute a fuzzy set value for 
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each of a plurality of vertical fuzzy sets each having a selected weighting function 
applicable to a selected group of rows within the object segment, with the rows 
grouped according to position along the vertical axis in the object segment; and, 

the background modeling apparatus is adapted to isolate from a current 
frame the at least one segment of the current frame utilizing the video object plane 
extracted from the current frame; and 

the horizontal fuzzy sets further comprising: 

a left extreme fuzzy set having a weighting function that is a constant value 
for a first selected number of columns from the left border of the object 
segment toward the vertical axis of the object segment and decreases to 
zero over a second selected number of columns toward the vertical axis if 
the object segment; 

a right extreme fuzzy set having a weighting function that is a constant 
value for a third selected number of columns from the right border of the 
object segment toward the vertical axis of the object segment and decreases 
to zero over a fourth selected number of columns toward the vertical axis of 
the object segment; and 

at least one horizontal internal fuzzy set having a weighting function that 
increases from zero to a peak value and decreases to zero as the columns 
contained in each such internal fuzzy set go from left to right across the 
object image; 

the vertical fuzzy sets further comprising: 

a bottom extreme fuzzy set having a weighting function that is a constant 
value for a fifth selected number of rows from the bottom border of the 
object segment toward the horizontal axis of the object segment and 
decreases to zero over a sixth selected number of rows toward the 
horizontal axis if the object segment; 

a top extreme fuzzy set having a weighting function that is a constant value 
for a seventh selected number of columns from the top border of the object 
segment toward the horizontal axis of the object segment and decreases to 
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zero over an eighth selected number of rows toward the horizontal axis of 
the object segment; and, 

at least one vertical internal fuzzy set having a weighting function that 
increases from zero to a peak value and decreases to zero as the rows 
5 contained in each such vertical internal fuzzy set go from bottom to top 

across the object image; and, 

the at least one segment of the current frame is a bounding box. 

4 1 . A video detection apparatus adapted to detect and follow movement of a 
1 0 predefined object of interest within the video data of the output of a video camera 
encoded in frame-wise data, each frame occurring at sequential time intervals, and 
including the segmentation of at least one portion of the frame-wise data, an object 
segment, as potentially including the preselected object of interest, and including 
vertical edge pixels and horizontal edge pixels detected to be present within the 
1 5 object segment, comprising; 

an edge symmetry detection unit comprising: 

a vertical symmetry detector adapted to compute the vertical symmetry of 
the set of vertical edge pixels and the set of horizontal edge pixels in each 
column about a horizontal axis passing through the object segment, and to 

2 0 produce a vertical symmetry value for columns within the object segment; 

a horizontal symmetry detector adapted to compute the horizontal 
symmetry of the set of vertical edge pixels and set of horizontal edge pixels 
in each row about a vertical axis passing through the object segment, and to 
produce a horizontal symmetry value for rows within the object segment; 

2 5 and, 

an edge symmetry feature set computation apparatus adapted to compute an 
edge symmetry feature set for the object segment based upon the vertical 
symmetry values and the horizontal symmetry values; and, 
a background modeling unit comprising: 
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a background modeling apparatus adapted to isolate from a current frame at 
least one segment of the current frame, each of the at least one segments 
containing a possible object of interest within the current frame, on a frame- 
wise basis, and to provide as an output each of the at least one segments; 
an object of interest classifying apparatus adapted to determine whether or 
not any segment of the at least one segments output from the background 
modeling apparatus includes a object of interest and to characterize any 
such segment as an object segment, utilizing at least in part the edge 
symmetry feature set; 

an object segment tracking apparatus adapted to track the location within 
the current frame of any object segment, and to determine a projected 
location of the object segment in a subsequent frame, and to provide the 
background modeling apparatus with the projected location in the 
subsequent frame of the object segment; 

wherein the background modeling apparatus is adapted to provide as its 
output the projected location of the object segment in the subsequent frame 
in lieu of a segment from the subsequent frame determined by the 
background modeling apparatus to contain a possible object of interest or in 
the event that the background modeling apparatus is unable in the 
subsequent frame to locate any segment containing a possible object of 
interest; and, 

the object segment tracking apparatus further comprising: 
a matching apparatus adapted to match the location of an object segment in 
the current frame to one of a plurality of projected locations of the object 
segment in the current frame, which projections are based upon the location 
of the respective object segment in at least one prior frame; 
a track provider adapted to receive and store the location of an object segment in 
the current frame and over a plurality of prior frames and adapted to provide the 
projections of the location of the object segment in a subsequent frame based upon 
the locations of the object segments in the current frame and the plurality of prior 
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frames; and, 

the vertical symmetry detector is adapted to compute the vertical symmetry 
value of at least one of the set of horizontal edges or the set of vertical edges in a 
given column / as the ratio of the number of horizontal edge pixels or vertical edge 
5 pixels, respectively, in each column / on opposing sides of the horizontal axis, 
independent of which side has more horizontal edge pixels or vertical edge pixels, 
respectively, expressed as a number between 0, if there are no horizontal edge 
pixels or no vertical edge pixels, respectively, on either side of the horizontal axis 
and 1 .0 if there are, respectively, an equal number of horizontal edge pixels or 

1 0 vertical edge pixels, respectively on both sides of the horizontal axis; and, 

the horizontal symmetry detector is adapted to compute the horizontal 
symmetry value of at least one of the set of horizontal edges or the set of vertical 
edges in a given row / as the ratio of the number of horizontal edge pixels or 
vertical edge pixels, respectively, in each row/ on opposing sides of the vertical 

15 axis, independent of which side has more horizontal edge pixels or vertical edge 
pixels, respectively, expressed as a number between 0, if there are no horizontal 
edge pixels or no vertical edge pixels, respectively, on either side of the horizontal 
axis and 1 .0 if there are, respectively, an equal number of horizontal edge pixels or 
vertical edge pixels, respectively on both sides of the vertical axis; 

2 0 the background modeling apparatus is adapted to provide as its output the 

projected location of the object segment in the subsequent frame in order to 
override any conflict in the labeling of pixels in the subsequent frame as part of a 
segment containing a possible object of interest; 

the feature set computation apparatus further comprising: 

2 5 a horizontal fuzzy set computation apparatus adapted to compute a fuzzy 

set value for each of a plurality of horizontal fuzzy sets each having a 
selected weighting function applicable to a selected group of columns 
within the object segment, with the columns grouped according to position 
along the horizontal axis in the object segment; and, 

3 0 a vertical fuzzy set computation apparatus adapted to compute a fuzzy set value for 
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each of a plurality of vertical fuzzy sets each having a selected weighting function 
applicable to a selected group of rows within the object segment, with the rows 
grouped according to position along the vertical axis in the object segment; and, 

the background modeling apparatus is adapted to isolate from a current 
frame the at least one segment of the current frame utilizing the video object plane 
extracted from the current frame; and 

the horizontal fuzzy sets further comprising: 

a left extreme fuzzy set having a weighting function that is a constant value 
for a first selected number of columns from the left border of the object 
segment toward the vertical axis of the object segment and decreases to 
zero over a second selected number of columns toward the vertical axis if 
the object segment; 

a right extreme fuzzy set having a weighting function that is a constant 
value for a third selected number of columns from the right border of the 
object segment toward the vertical axis of the object segment and decreases 
to zero over a fourth selected number of columns toward the vertical axis of 
the object segment; and 

at least one horizontal internal fuzzy set having a weighting function that 
increases from zero to a peak value and decreases to zero as the columns 
contained in each such internal fuzzy set go from left to right across the 
object image; 

the vertical fuzzy sets further comprising: 

a bottom extreme fuzzy set having a weighting function that is a constant 
value for a fifth selected number of rows from the bottom border of the 
object segment toward the horizontal axis of the object segment and 
decreases to zero over a sixth selected number of rows toward the 
horizontal axis if the object segment; 

a top extreme fuzzy set having a weighting function that is a constant value 
for a seventh selected number of columns from the top border of the object 
segment toward the horizontal axis of the object segment and decreases to 
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zero over an eighth selected number of rows toward the horizontal axis of 
the object segment; and, 

at least one vertical internal fuzzy set having a weighting function that 
increases from zero to a peak value and decreases to zero as the rows 
5 contained in each such vertical internal fuzzy set go from bottom to top 

across the object image; 

the at least one segment of the current frame is a bounding box; and, 
the columns associated with each of the horizontal fuzzy sets are 

overlapping into respective adjacent horizontal fuzzy sets; and, 
1 0 the rows associated with each of the vertical fuzzy sets are overlapping into 

respective adjacent vertical fuzzy sets. 

42. A video detection apparatus adapted to detect and follow movement of a 
predefined object of interest within the video data of the output of a video camera 
1 5 encoded in frame-wise data, each frame occurring at sequential time intervals, and 
including the segmentation of at least one portion of the frame-wise data, an object 
segment, as potentially including the preselected object of interest, and including 
vertical edge pixels and horizontal edge pixels detected to be present within the 
object segment, comprising: 
2 0 an edge symmetry detection unit comprising: 

a vertical symmetry detector adapted to compute the vertical symmetry of 
the set of vertical edge pixels and the set of horizontal edge pixels in each 
column about a horizontal axis passing through the object segment, and to 
produce a vertical symmetry value for columns within the object segment; 

2 5 a horizontal symmetry detector adapted to compute the horizontal 

symmetry of the set of vertical edge pixels and set of horizontal edge pixels 
in each row about a vertical axis passing through the object segment, and to 
produce a horizontal symmetry value for rows within the object segment; 
and, 

3 0 an edge symmetry feature set computation apparatus adapted to compute an 



-88- 

c : Bill \pa tent applications draf ted\XLaboratories\ 104 4-4 13-01 . doc 



edge symmetry feature set for the object segment based upon the vertical 
symmetry values and the horizontal symmetry values; and, 
a background modeling unit comprising: 

a background modeling apparatus adapted to isolate from a current frame at 
least one segment of the current frame, each of the at least one segments 
containing a possible object of interest within the current frame, on a frame- 
wise basis, and to provide as an output each of the at least one segments; 
an object of interest classifying apparatus adapted to determine whether or 
not any segment of the at least one segments output from the background 
modeling apparatus includes a object of interest and to characterize any 
such segment as an object segment, utilizing at least in part the edge 
symmetry feature set; 

an object segment tracking apparatus adapted to track the location within 
the current frame of any object segment, and to determine a projected 
location of the object segment in a subsequent frame, and to provide the 
background modeling apparatus with the projected location in the 
subsequent frame of the object segment; 

wherein the background modeling apparatus is adapted to provide as its 
output the projected location of the object segment in the subsequent frame 
in lieu of a segment from the subsequent frame determined by the 
background modeling apparatus to contain a possible object of interest or in 
the event that the background modeling apparatus is unable in the 
subsequent frame to locate any segment containing a possible object of 
interest; and, 

the object segment tracking apparatus further comprising: 
a matching apparatus adapted to match the location of an object segment in 
the current frame to one of a plurality of projected locations of the object 
segment in the current frame, which projections are based upon the location 
of the respective object segment in at least one prior frame; 
track provider adapted to receive and store the location of an object segment in 
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the current frame and over a plurality of prior frames and adapted to provide the 
projections of the location of the object segment in a subsequent frame based upon 
the locations of the object segments in the current frame and the plurality of prior 
frames; and, 

5 the vertical symmetry detector is adapted to compute the vertical symmetry 

value of at least one of the set of horizontal edges or the set of vertical edges in a 
given column / as the ratio of the number of horizontal edge pixels or vertical edge 
pixels, respectively, in each column / on opposing sides of the horizontal axis, 
independent of which side has more horizontal edge pixels or vertical edge pixels, 
10 respectively, expressed as a number between 0, if there are no horizontal edge 

pixels or no vertical edge pixels, respectively, on either side of the horizontal axis 
and 1 .0 if there are, respectively, an equal number of horizontal edge pixels or 
vertical edge pixels, respectively on both sides of the horizontal axis; and, 

the horizontal symmetry detector is adapted to compute the horizontal 
1 5 symmetry value of at least one of the set of horizontal edges or the set of vertical 
edges in a given row j as the ratio of the number of horizontal edge pixels or 
vertical edge pixels, respectively, in each row j on opposing sides of the vertical 
axis, independent of which side has more horizontal edge pixels or vertical edge 
pixels, respectively, expressed as a number between 0, if there are no horizontal 
2 0 edge pixels or no vertical edge pixels, respectively, on either side of the horizontal 
axis and 1.0 if there are, respectively, an equal number of horizontal edge pixels or 
vertical edge pixels, respectively on both sides of the vertical axis; 

the background modeling apparatus is adapted to provide as its output the 
projected location of the object segment in the subsequent frame in order to 
2 5 override any conflict in the labeling of pixels in the subsequent frame as part of a 
segment containing a possible object of interest; 

the feature set computation apparatus further comprising: 
a horizontal fuzzy set computation apparatus adapted to compute a fuzzy 
set value for each of a plurality of horizontal fuzzy sets each having a 
30 selected weighting function applicable to a selected group of columns 
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within the object segment, with the columns grouped according to position 

along the horizontal axis in the object segment; and, 
a vertical fuzzy set computation apparatus adapted to compute a fuzzy set value for 
each of a plurality of vertical fuzzy sets each having a selected weighting function 
applicable to a selected group of rows within the object segment, with the rows 
grouped according to position along the vertical axis in the object segment; and, 

the background modeling apparatus is adapted to isolate from a current 
frame the at least one segment of the current frame utilizing the video object plane 
extracted from the current frame; and 

the horizontal fuzzy sets further comprising: 

a left extreme fuzzy set having a weighting function that is a constant value 
for a first selected number of columns from the left border of the object 
segment toward the vertical axis of the object segment and decreases to 
zero over a second selected number of columns toward the vertical axis if 
the object segment; 

a right extreme fuzzy set having a weighting function that is a constant 
value for a third selected number of columns from the right border of the 
object segment toward the vertical axis of the object segment and decreases 
to zero over a fourth selected number of columns toward the vertical axis of 
the object segment; and 

at least one horizontal internal fuzzy set having a weighting function that 
increases from zero to a peak value and decreases to zero as the columns 
contained in each such internal fuzzy set go from left to right across the 
object image; 

the vertical fuzzy sets further comprising: 

a bottom extreme fuzzy set having a weighting function that is a constant 
value for a fifth selected number of rows from the bottom border of the 
object segment toward the horizontal axis of the object segment and 
decreases to zero over a sixth selected number of rows toward the 
horizontal axis if the object segment; 
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a top extreme fuzzy set having a weighting function that is a constant value 
for a seventh selected number of columns from the top border of the object 
segment toward the horizontal axis of the object segment and decreases to 
zero over an eighth selected number of rows toward the horizontal axis of 
the object segment; and, 

at least one vertical internal fuzzy set having a weighting function that 
increases from zero to a peak value and decreases to zero as the rows 
contained in each such vertical internal fuzzy set go from bottom to top 
across the object image; 

the at least one segment of the current frame is a bounding box; 

the columns associated with each of the horizontal fuzzy sets are 
overlapping into respective adjacent horizontal fuzzy sets; and, 

the rows associated with each of the vertical fuzzy sets are overlapping into 
respective adjacent vertical fuzzy sets; and, 

the feature set computation apparatus further comprising a fuzzy set edge 
value score computing apparatus adapted to compute a fuzzy set edge value based 
upon the respective horizontal edge value or vertical edge value for the respective 
column or row and the respective horizontal fuzzy set weighting function value for 
the respective column or the respective vertical fuzzy set weighting function for the 
respective row. 
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